Evaluation and ranking of manufacturing line non-numeric information

ABSTRACT

An apparatus and method for quantitatively ranking the performance of each attribute contributing to a manufacturing process. Various signals representing the measure (i.e., quality or yield) of a manufacturing run and representing the attributes that contributed to each manufacturing run are recorded. An iterative process is commenced whereby numeric weights are assigned to each attribute. In a first iterative step, the weight of each attribute is determined to be the weighted average of the measure of each manufacturing run to which that attribute contributed. In subsequent iteration steps, a refined weight for each attribute is determined by computing the ratio between the normalized measure and the product of all other weights associated with that manufacturing run. The iterations are continued until the weights determined for each attribute converge or become self consistent. In this manner, the performance of non-numeric or attribute parameters such as operators and tools can be measured and defined in terms of a numeric quantity.

This application is a continuation of application Ser. No. 08/217,272,filed Mar. 24, 1994, now U.S. Pat. No. 5,479,361, which is acontinuation of application Ser. No. 07/843,048, filed Feb. 27, 1992,now U.S. Pat. No. 5,351,202, issued Sep. 27, 1994.

TECHNICAL FIELD

A portion of the disclosure of this patent document contains materialwhich is subject to copyright protection. The copyright owner has noobjection to the facsimile reproduction by anyone of the patentdisclosure, as it appears in the Patent and Trademark Office patentfiles or records, but otherwise reserves all copyright rightswhatsoever.

The present invention is concerned with improvements in manufacturingline processes. More particularly, the present invention relates toimproving the determination and evaluation of manufacturing lineinformation. Still more particularly, the present invention relates to amethod for evaluating and ranking non-numeric manufacturing lineinformation to evaluate the performance of the line and ultimatelyimprove the quality of products turned out by the line.

BACKGROUND ART

In a manufacturing line of almost any type, there are a number offactors that control and/or affect the quality of the manufacturingprocess and of the product produced by the process. Some of thecontrolling or operative factors are numeric, such as temperature, time,distance between stations, tool settings, etc. These numeric factors areeasily quantifiable. Other factors are non-numeric and are not readilyquantifiable. Such factors, called "attribute" information, include thespecific machine or tool, the particular operator or combination ofoperators, the work-shift, quality of starting materials, and the like.Each of the numeric and non-numeric factors controls or has an effect onthe quality of the final product. For adequate control of themanufacturing line, it is desirable to optimize its parametrics (numericinformation) as well as its attributes (non-numeric information).

It would be very useful to be able to quantify the non-numeric factors.By doing so, a line manager or supervisor could then make appropriateadjustments to the manufacturing line to obtain improved product yieldsor quality.

As an illustration of the problem, consider the manufacturing of an MLC(multi-layer ceramic) package. The manufacturing process consists ofstacking many layers of ceramic "sheets". The stacked sheets are thensintered. The process involves multiple intermediate or startingproducts and materials, different tools, and multiple operators. Theparametrics include the number of sheets in a stack, the number ofstacks in the line, and the sintering temperature and times. Theattributes of the MLC manufacturing process include such things as theidentities of the operators (OP-ID), the particular stacking tool(s)used on the line, the particular sintering chamber(s), the particularmeasuring tools, the work-shift(s) (that is, whether the line is runningon the day shift, the night shift or the "graveyard" shift), the vendorsfrom whom starting materials are purchased, etc.

Process yield is the primary performance measure. It is desired toevaluate the performance of all the tools and operators used on the MLCmanufacturing line so that "bad" tools can be fixed or replaced and"poor" operators can be retrained. The evaluating mechanism shoulditself yield good results even when not all the combinations ofattributes occur in reality. In addition, the mechanism should resolveconflicting "opinions" correctly.

The present inventors are aware of a number of known systems whichpurport to control manufacturing processes. To their knowledge, all ofthe known systems operate on numeric data only. Such known systems arerepresented by the following issued U.S. patents:

    ______________________________________                                        U.S. Pat. No. 4,089,056                                                                     METHOD AND AUTOMATED                                                          EQUIPMENT FOR THE TRACK-                                                      ING, CONTROL AND SYNTHESIZ-                                                   ING OF MANUFACTURING                                                          PERFORMANCE FIGURES.                                            U.S. Pat. No. 4,115,848                                                                     METHOD AND SYSTEM FOR                                                         CONTROLLING PLANTS.                                             U.S. Pat. No. 4,328,556                                                                     CONTROL SYSTEM OF PLANTS                                                      BY MEANS OF ELECTRONIC                                                        COMPUTERS.                                                      U.S. Pat. No. 4,604,718                                                                     COMPUTER SIMULATION                                                           SYSTEM.                                                         U.S. Pat. No. 4,858,154                                                                     INTERLABORATORY QUALITY                                                       ASSURANCE PROGRAM.                                              U.S. Pat. No. 4,870,590                                                                     MANUFACTURING LINE                                                            CONTROL SYSTEM.                                                 U.S. Pat. No. 5,008,842                                                                     METHOD OF SEQUENTIAL                                                          MONITORING AND APPARATUS                                                      FOR PRACTICING THE SAME.                                        U.S. Pat. No. 5,047,947                                                                     METHOD OF MODELLING THE                                                       ASSEMBLY OF PRODUCTS TO                                                       INCREASE PRODUCTION YIELD.                                      ______________________________________                                    

The known systems exemplified by the above-listed patent disclosures allhave at least one thing in common. None of them takes into account thenon-numeric components of the manufacturing line process whichcontribute to the product yield of the line.

For example, the '056 patent describes a system for generating codedinformation from a plurality of electromechanical transducer unitsassociated with respective machines to be supervised. Control of themanufacturing line is based on the data collected from the transducerunits.

The '848 and '556 patents describe related process control systems andmethods in which the operation of the plant (e.g., an electric powergenerating plant, a chemical processing plant, etc.) is controlled onthe basis of tables stored in a computer and variations from normsdetected by process input devices. The system is continuously controlledon the basis of comparing measured parameters with information stored inthe tables.

The '718 patent describes a system for computer simulating the operationof a user-specified automated manufacturing and/or materials handlingfacility. The facility designer specifies various computer-controlledcomponents and operating parameters which are to be tested. The designerselects from lists of predetermined components and operating parameters.The computer then creates an operational prototype model of the actualsystem.

The '154 patent describes a technique for statistically evaluating theperformance, precision, and/or accuracy of one of a plurality of likeinstruments. The technique calls for comparing the performance of eachinstrument located in physically disparate locations. The patent isspecifically directed to evaluating the performance of blood analyzersas part of an Interlaboratory Quality Assurance Program (IQAP).

The '590 patent describes a manufacturing line control system in whichvarious analog, digital and status signal information issuing from aproduct on the line is evaluated. Control signals are then issued to themanufacturing line based on the detected product information.

The '842 patent describes a method and apparatus for sequentiallymonitoring the state of a working manufacturing line. As with otherknown systems, sensors on the line detect various parameters on themanufacturing line, including emulsion application, web density, webwinding, etc. As with other known systems, all of the parametersmeasured are numeric quantities.

The '947 patent describes a method for increasing the production ormanufacturing yield of a product by the application of mathematicalmodeling techniques on the statistical distributions of major componentsof the product to maximize the production yield of the products.Initially, the statistical characteristics of significant performancespecifications of each of the major components of the product areestablished for each manufacturing lot of that component. Thestatistically probable performance of products assembled from differentmanufacturing lots is evaluated to assess the performance sensitivity ofthe assembled product to different combinations of the components fromdifferent manufacturing lots.

In order to maximize the yield from a given manufacturing line, it isdesirable to account for non-numeric, as well as numeric, factors. It iswell known empirically that non-numeric factors can play a significantrole in product yield or quality. For example, studies have shown thatautomobiles manufactured on a Monday are statistically likely to have ahigher percentage of defects than those coming off the assembly line ona Tuesday or Wednesday. No two milling machines are identical. Twoassembly lines using two different milling machines will produceproducts that are not exactly identical. This concept carries over toall aspects of a manufacturing process. While absolute identity offinished products is not always essential, there are times, perhaps insome critical processes, when it is desirable to come as close aspossible to absolute uniformity of finished product, whether from asingle manufacturing line or from a plurality of similar lines.

The present invention was developed to take into account non-numericfactors which affect manufacturing lines. The present inventionovercomes the deficiencies of known systems in this respect.

DISCLOSURE OF INVENTION

The present invention provides an apparatus and method for assigning aquantitative ranking to non-numeric attributes that contribute to theprocess of a manufacturing line. The present invention can be used toquantitatively measure the performance of non-numeric, or attribute,data including the performance of particular tools or machines on themanufacturing line, the performance of particular operators on themanufacturing line, and the effects of external conditions such asweather and shift changes.

The technique of the present invention is to normalize the performancemeasure of a particular manufacturing line run to a value between 0and 1. Performance measure typically defined in terms of process unit orquality. Each measure can then be written as a product of all the weightresponsible for that measure. One weight is contributed by eachattribute and the range of attribute weights is 0 to 1,1 being perfect.

The present invention comprises a technique for deconvoluting theindividual attribute weights associated with a given process, thuspermitting one to evaluate the significance of each attribute associatedwith the entire manufacturing process, independent of all otherattributes.

In the technique of the present invention, the measures of severalmanufacturing runs and the attributes associated with each run arerecorded. The measure for each run is then normalized to a numberbetween ε and 1 where ε is very near 0, thus expressing each measure asa fraction of the perfect measure of 1. In a first iteration, the weightof each attribute is taken as the weighted average of the measures ofthe manufacturing runs to which that attribute contributed. Insubsequent iterations, new weights for each attribute are computed asthe ratio between the normalized measure of the manufacturing run andthe product of all other associated weights. The iterations arecontinued until the refined weights of each attribute converge or becomeself-consistent. In this manner, given a set of measured performances,the individual contributions of each attribute can be determined, theproduct of which best describes the total measure. For each non-numericattribute, a numeric measure can now be assigned.

The foregoing and other objects, features and advantages of theinvention will be apparent from the following more particulardescription of preferred embodiments of the invention, as illustrated inthe accompanying drawings.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a flowchart illustrating the iterative process of the presentinvention.

FIG. 2 is a block diagram illustrating the configuration of the presentinvention.

FIG. 3 is an example of a graphical representation of theself-consistent weighted averages of each of the individual attributes.

BEST MODE FOR CARRYING OUT THE INVENTION

1. Overview of the Present Invention

The performance of a manufacturing line depends on a combination ofnumeric and non-numeric information or data. The numeric data includesquantifiable parametrics such as tool settings, temperatures, processtimes, etc. Non-numeric, or attribute, information includes qualitativeinformation such as the performance of a particular machine or operatorwith respect to other machines or operators performing the samefunction(s), the particular shift during which a product is produced oron which an operator works, the day of the week, the weather, thequality of materials received from a particular vendor, etc.

When adjusting the manufacturing line process, quality of the productand yield of the process should be maximized. This requires properadjustment of both the parametrics and attributes of the manufacturingline.

Because of the differences in non-numeric, or attribute data, twoproducts from the same manufacturing line may vary. For example, anautomobile assembly line may produce a first automobile which isvirtually trouble-free and immediately follow with production of a"lemon." The reason for this is although parametrics can be closelycontrolled, individual attributes may, and usually do, vary dramaticallywithin a class of attributes. Performance of seemingly identical toolsused in the assembly of the automobile may vary as does the performanceof each individual operator. Moreover, placing particular operators on aparticular shift or rotating shifts may affect quality and yield of themanufacturing process.

Manufacturing processes often have numerous independent attributesworking to produce a single continuous measure (e.g. yield). There areoften numerous permutations of combinations of individual attributescontributing to the process. This poses a serious impediment toevaluation of each individual attribute. For example a good operatorworking on a bad machine may produce the same result as a bad operatorworking on a good machine.

Therefore, it is desired to evaluate each individual attribute andascribe a quantitative value to that attribute which is a measure of itsperformance independent of the influence of other attributes. To dothis, the measure of each individual attribute must be extracted, ordeconvoluted, from the other attributes.

As an illustration, one can consider the manufacturing of an MLCpackage. The manufacturing process consists of stacking many layers ofceramic "sheets" which are then sintered. The process involves differentproducts, tools, and operators, often in different combinations of theseindividual attributes. The performance measure in this case would beprocess yield. It is desirable to evaluate the performance of each tooland operator associated with the manufacturing line so that "bad" toolscan be fixed or replaced and "poor" operators can be retrained. Theevaluating mechanism should yield good results even when not allcombinations of attributes occur in reality. In addition, the mechanismshould resolve conflicting "opinions" correctly.

Each yield of the MLC manufacturing process contains contributions fromindividual members of each attribute class. The problem becomes a matterof deconvolution of the effects (i.e., the weights) of individualattributes.

Each performance measure can be normalized to a value between 0 and 1(e.g., Yield becomes a fraction between 0 and 1). Each measure can thenbe written as a product of all the weights responsible for that measure.One weight is contributed by each attribute, and the range of attributeweights is 0 to 1, 1 being perfect. Therefore, each of the weights wouldexactly equal the resultant performance measure if all other attributeswere perfect (i.e., with weights of "1").

The present invention comprises a technique for deconvoluting theindividual attribute weights associated with a given process. Thus, thetechnique of this invention permits one to evaluate the significance ofeach separate attribute associated with the entire manufacturingprocess. More specifically, by using the technique of the presentinvention, it is possible to rate each of the tools used on a given lineto determine which need to be replaced or repaired; it is also possibleto evaluate each of the operators who work on the line to determinewhich, if any, require retraining to improve their performance.

The present invention provides a technique for ascribing a measure, in amathematical sense, to each individual attribute. For each attribute, anumerical value, or weight, between 0 and 1 is determined. This weightis a measure of the individual attribute's performance relative to otherindividual attributes in the attribute class.

The technique of this invention, sometimes called the "self-consistent"or "convergence" approach, is adapted from a well-known technique inatomic physics used to solve the many-body problem of a multi-electronatom. The atomic problem is solved via iterations over products ofsingle-electron wave functions which are derived from the solvableproblem of a single electron moving under the influence of the weightedaverage of all other electrons (and the nucleus). As used in the atomicphysics technique and in that of the present invention,"self-consistency" means that the iterated set of the wave functions ofall electrons eventually converges such that it is not significantlydifferent from the previous set. Although no formal proof of convergenceexists in spite of repeated efforts, it has converged in all known casesin which the technique has been used. (See, for example, Slater, JohnC., "The Self-consistent Field for Molecules and Solids," McGraw Hill,1974).

The method of the present invention extends the known approach to enableits adoption in attribute analysis for manufacturing lines andprocesses. The attributes are somewhat comparable to the separate atomicwavefunctions; the performance measure represents the total atomicwavefunction. The present inventive technique goes beyond the atomicphysics technique by identifying a finite set of members of eachattribute class, from which one is used in each given measure.

FIG. 1 is a flowchart illustrating the technique of the presentinvention. Referring to FIG. 1, in a step 102, the measures of severalmanufacturing runs are recorded. As previously noted, measure can beexpressed in terms of yield. Also recorded are the attributes associatedwith each run--i.e., specific tools used or particular operatorsinvolved. In a step 104, the measure for each run is normalized to anumber between epsilon and 1, where epsilon is very near 0. This isaccomplished by subtracting the value of epsilon from the measure anddividing this number by the maximum measure. In this manner, each actualmeasure can be expressed as a fraction of the perfect measure of 1.

In a step 106, it is determined in which of the recorded runs the firstattribute participated. These runs are selected to determine the valueor weight of the attribute. In a step 108A, all the other attributesthat make up each selected run are assumed to be "perfect," and aretherefore not considered. In other words, the measure of each run isattributed entirely to the first attribute. For example, if a givenmeasure is 0.785 the value of the first attribute for that run is takenas 0.785. In a step 110, the weighted average of all measures to whichthe attribute contributed is determined. This weighted average for thefirst attribute, provides an initial estimate of the weight of theattribute. In a step 112, steps 106 through 110 are repeated for each ofthe other attributes. In this manner, an initial estimate of the weightof each attribute is made.

In a step 114, steps 106 through 112 are repeated, this time following astep 108B. In step 108B, the measure of each run is not ascribedentirely to the contribution of the attribute under consideration. Now,each measure has k weights associated with it. These weights are thoseof each of the k attributes associated with that measure as determinedin the previous iteration. A new weight for the attribute is computed asthe ratio between the normalized measure and the product of all otherassociated weights. Therefore, the estimate of the attribute weightobtained in step 110 is a refined estimate.

Step 114 is repeated until the refined weights of each attributeconverge or become "self-consistent." To determine when they converge,one compares the current attributes' weights to those weights obtainedfor that attribute in the previous iteration. This can be done bycomputing the sum-of-squares of the differences of the two sets, or bycomputing the maximum of the absolute difference of the two sets.

Each attribute now has a weight which is a weighted measure of allmeasures in which it is involved, and which is termed convergent.

In effect, given a set of measured performances, the individualcontributions of the attributes can be determined, the product of whichbest describes (through convergence) the total measure. For eachnon-numeric attribute, a numeric measure can now be assigned.

2. Detailed Description of the Preferred Embodiment

As stated above, the basic technique is to compute a first-guess weightfor each attribute while holding the others constant, and then performiterations using the newly computed weight for each attribute, theiterations continuing until the attribute weights converge. Thepreferred embodiment of the present invention will now be described.

A block diagram of the present invention is illustrated in FIG. 2.Referring now to FIG. 2, a manufacturing line 202 receives inputmaterials 204 to be processed. Input materials 204 may come from asupply of raw materials, a previous process step, or the like.Attributes 1-K, generally designated 206, perform functions relating tothe processing of materials 204 in the process of manufacturing line202. These attributes 206 can include tools and machinery, variousoperators, other variables such as day of the week or shift number, andeven various manufacturing processes. Each attribute 206, i.e., eachoperator, each tool, each process, etc. has its own unique attributesignal ID 208. This ID signal 208 is a physical signal that uniquelyidentifies each attribute 206 contributing to the manufacturing process.

The manufacturing line 202 produces a completed product 210, or aproduct ready to move on to the next phase of processing. A measuresignal 212 provides a quantitative measure of the quality of themanufacturing run. Measure signal 212 is a physical signal that canrepresent the measure of the run in terms of various metrics such asoutput yield and output quality. A processor 214 receives measuresignals 212 and attribute ID signals 208. Measure signal 212 for eachmanufacturing run, and ID signals 208 of attributes 206 that contributedto that manufacturing run are stored in a database 216. Additionalattribute ID signals 208 can be provided to processor 214 and database216 for each manufacturing run. These additional attribute ID signals208 can be provided through various means such as manual entry via aremote terminal 218, automated or semi-automated entry via a remote workstation 220.

Referring now to FIGS. 1 and 2, the technique of the present inventionwill now be described. In step 102 the processor 214 receives measuresignals 212 which indicate the output measure 210 of manufacturing line202 for each manufacturing run. These measure signals 212 are stored ondatabase 216. Processor 214 also receives ID signals 208 indicating eachof the attributes that contributed to that particular manufacturing run.These ID signals 208 are also stored in database 216.

In step 104, processor 214 normalizes the measure of each run to anumber between ε and 1 where ε is very near 0. Processor 214accomplishes this normalization by subtracting the value of ε from themeasure signal 212 and dividing this number by the maximum measuresignal 212. Once a suitable number of manufacturing runs have beenrecorded, the weighting process can begin. Statistically, a greaternumber of runs recorded to which an attribute contributed provides amore accurate weight that can be ascribed to that attribute. An editingstep may be added to edit runs containing apparently anomalous datapoints, i.e., runs whose yields are above or below specified bounds, aredeleted from the database.

In step 106, processor 214 selects all the manufacturing runs in whichthe first attribute participated. In step 108A, processor 214 assumesthat all other attributes are perfect and temporarily assigns them aweight of 1. As a result, the value of measure signal 212 for each runto which attribute 1 contributed is ascribed entirely to attribute 1. Instep 110, processor 214 determines the weighted average of all measuresignals 212 to which attribute 1 contributed. This weighted average forthe first attribute provides an initial attribute estimate of the weightof the first attribute. Processor 214 stores this initial estimate ondatabase 216.

In step 112, steps 106-110 are repeated for each of the attributes 2-K.In this manner, an initial estimate of the weight of each attribute ismade and stored.

In step 114, steps 106-112 are repeated, this time following step 108B.In step 108B, the measure of each run is not ascribed entirely to thecontribution of the attribute under consideration as was done in step108A. Now, each measure has K weights associated with it. These weightsare those of each of the K attributes associated with that measure asdetermined in the previous iteration. Processor 214 computes a newweight for the attribute as the ratio between the normalized measuresignal 212 and the product of all other associated weights 1-K.Therefore, the estimate of the attribute weight obtained in step 110 ofthis iteration is a refined estimate. This refined estimate is stored indatabase 216. In step 112, processor 214 repeats steps 106, 108B and 110for each of the attributes, thereby obtaining and storing a refinedestimate for each attribute.

In step 114, processor 214 repeats steps 106-112 until the refinedweights of each attribute converge or become "self-consistent." Todetermine when the weights converge, one compares the current weight ofeach attribute to the weight obtained for that attribute in the previousiteration. This can be done by computing the sum of squares of thedifferences of the two sets, or by computing the maximum of the absolutedifference of the two sets.

Processor 214 generates and prints reports and graphics based on theinformation contained in database 216. Reports selected can includeinformation such as alpha numeric representations of measure signals 212and ID signals 208 associated with each manufacturing run, measuresignals and ID signals 208 for each individual attribute, or numerousother combinations of the information stored in database 216.Alternatively, this information can be displayed on a display monitor222 or sent to work station 220 for further manipulation and/or display.

Table 1 provides an example of physical signal information that may bestored in database 216.

                  TABLE I                                                         ______________________________________                                        YIELD   @MEFAM    @TOOL     @SCREENER @OPID                                   ______________________________________                                        0.99    143       ORAF0000  28        637                                     0.99    141       ORAF0000  23        225                                     0.97    145       ORAG0000  28        D63                                     0.86    143       ORAG0000  22        261                                     0.94    143       ORAF0000  28        637                                     0.96    145       ORAG0000  22        261                                     0.92    153       ORAF0000  29        604                                     0.92    141       ORAG0000  21        848                                     0.96    143       ORAF0000  29        560                                     0.91    145       ORAG0000  20        426                                     0.93    143       ORAF0000  28        637                                     0.88    143       ORAF0000  22        233                                     0.92    143       ORAF0000  28        D63                                     0.99    143       ORAF0000  22        595                                     0.86    143       ORAF0000  26        568                                     0.92    153       ORAG0000  27        C02                                     0.91    153       ORAF0000  27        426                                     0.98    142       ORAE0000  21        848                                     0.98    143       ORAE0000  28        637                                     0.98    142       ORAF0000  21        848                                     0.81    143       ORAE0000  21        C79                                     0.90    143       ORAG0000  26        890                                     0.96    145       ORAG0000  29        604                                     0.99    143       ORAF0000  26        215                                     0.93    143       ORAG0000  29        090                                     O.95    143       ORAF0000  26        890                                     0.95    143       ORAF0000  26        568                                     0.90    155       ORAE0000  20        225                                     0.91    141       ORAE0000  28        231                                     0.97    141       ORAG0000  28        637                                     0.94    145       ORAE0000  29        604                                     0.96    145       ORAE0000  20        592                                     0.81    145       ORAG0000  29        090                                     0.94    143       ORAF0000  28        D63                                     0.97    155       ORAE0000  23        225                                     0.93    152       ORAG0000  28        231                                     0.95    143       ORAF0000  29        560                                     0.94    142       ORAG0000  20        181                                     0.97    143       ORAF0000  26        215                                     0.79    145       ORAG0000  22        261                                     0.94    152       ORAE0000  29        604                                     0.95    152       ORAG0000  28        231                                     0.95    143       ORAF0000  26        890                                     0.90    143       ORAE0000  26        426                                     0.97    143       ORAF0000  26        568                                     0.95    143       ORAF0000  26        A05                                     0.97    143       ORAG0000  28        231                                     0.94    155       ORAE0000  21        C02                                     0.87    155       ORAG0000  22        391                                     0.91    143       ORAF0000  28        231                                     0.85    143       ORAF0000  28        637                                     0.87    143       GRAG0000  20        181                                     0.86    141       ORAG0000  22        233                                     0.92    145       ORAE0000  21        848                                     6.86    145       ORAE0000  20        595                                     0.99    152       ORAG0000  26        568                                     0.95    143       ORAE0000  26        568                                     0.98    145       ORAE0000  21        680                                     0.98    155       ORAE0000  21        595                                     0.96    143       ORAG0000  26        A05                                     0.97    152       ORAF0000  20        181                                     0.95    145       ORAE0000  20        097                                     0.95    143       ORAF0000  29        152                                     0.86    143       ORAE0000  29        604                                     0.98    152       ORAF0000  20        004                                     0.94    143       ORAG0000  26        215                                     0.82    143       ORAF0000  23        225                                     1.00    155       ORAE0000  21        595                                     0.98    155       ORAE0000  22        233                                     1.00    155       ORAG0000  26        560                                     0.97    143       ORAF0000  23        225                                     0.94    153       ORAG0000  21        C09                                     0.98    153       ORAF0000  20        181                                     1.00    155       ORAE0000  28        231                                     0.98    152       ORAG0000  23        225                                     0.83    142       ORAE0000  26        215                                     0.90    145       ORAE0000  20        097                                     0.98    155       ORAE0000  27        371                                     0.96    155       ORAE0000  27        426                                     0.90    155       ORAG0000  21        595                                     0.98    155       ORAE0000  26        568                                     1.00    155       ORAG0000  26        215                                     0.99    155       ORAG0000  22        261                                     0.45    153       ORAE0000  26        890                                     0.96    142       ORAE0000  26        637                                     0.94    142       ORAF0000  21        D63                                     0.95    155       ORAE0000  28        D63                                     0.97    155       ORAE0000  28        D63                                     0.89    143       ORAF0000  20        555                                     0.94    143       ORAE0000  23        555                                     0.97    145       ORAE0000  27        426                                     0.99    143       ORAE0000  28        090                                     0.96    143       ORAF0000  28        637                                     0.95    145       ORAE0000  26        215                                     1.00    I47       ORAF0000  26        568                                     0.99    151       ORAF0000  22        569                                     0.95    155       ORAE0000  22        261                                     0.99    152       ORAF0000  28        231                                     0.78    155       ORAE0000  21        848                                     0.93    145       ORAE0000  21        669                                     0.96    145       ORAE0000  21        C02                                     0.96    152       ORAE0000  23        349                                     0.98    143       ORAF0000  27        371                                     0.93    155       ORAE0000  21        848                                     0.91    145       ORAE0000  27        C02                                     0.98    151       ORAF0000  26        568                                     1.00    145       ORAE0000  22        261                                     0.99    142       ORAE0000  27        A18                                     1.00    155       ORAE0000  28        637                                     0.96    143       ORAF0000  26        568                                     0.98    145       ORAE0000  28        637                                     0.96    155       ORAE0000  28        637                                     0.98    152       ORAF0000  28        637                                     0.77    145       ORAE0000  20        181                                     0.86    153       ORAF0000  20        432                                     0.96    145       ORAE0000  27        B28                                     0.92    145       ORAE0000  22        231                                     0.92    145       ORAE0000  21        426                                     0.95    143       ORAF0000  28        231                                     0.84    145       ORAE0000  20        354                                     0.90    153       ORAF0000  29        604                                     0.97    145       ORAE0000  20        354                                     0.86    142       ORAF0000  23        255                                     0.98    143       ORAF0000  26        568                                     1.00    147       ORAF0000  27        C02                                     0.96    145       ORAE0000  26        568                                     0.99    145       ORAE0000  26        890                                     0.94    145       ORAE0000  27        371                                     0.98    145       ORAE0000  28        637                                     1.00    145       ORAE0000  21        848                                     0.95    152       ORAF0000  26        568                                     0.92    145       ORAE0000  22        215                                     0.91    143       ORAF0000  26        568                                     0.98    145       ORAE0000  28        637                                     0.94    145       ORAE0000  26        568                                     0.47    143       ORAF0000  21        848                                     0.82    145       ORAG0000  21        152                                     0.95    145       ORAE0000  28        637                                     0.95    145       ORAG0000  21        152                                     0.89    145       ORAG0000  21        231                                     0.90    145       ORAG0000  22        215                                     0.92    145       ORAE0000  22        090                                     0.95    145       ORAG0000  20        354                                     0.75    145       ORAE0000  22        569                                     0.79    145       ORAG0000  20        354                                     0.97    145       ORAE0000  28        D6J                                     0.92    142       ORAF0000  21        D63                                     0.89    145       ORAG0000  20        354                                     1.00    145       ORAE0000  26        890                                     1.00    145       ORAE0000  20        569                                     0.97    145       ORAG0000  21        152                                     1.00    145       ORAG0000  26        568                                     1.00    145       ORAG0000  22        215                                     0.97    145       ORAE0000  22        090                                     0.97    145       ORAG0000  28        637                                     0.97    145       ORAG0000  22        090                                     0.95    143       ORAF0000  28        D63                                     0.97    145       ORAE0000  27        371                                     0.91    145       ORAE0000  22        215                                     0.98    152       ORAF0000  20        592                                     1.00    145       ORAE0000  26        568                                     0.99    145       ORAG0000  29        604                                     0.89    145       ORAE0000  26        568                                     0.96    145       ORAG0000  20        354                                     0.98    153       ORAF0000  29        592                                     0.96    145       ORAG0000  22        261                                     0.96    145       ORAG0000  27        592                                     0.97    145       ORAE0000  27        C02                                     1.00    145       ORAE0000  21        152                                     1.00    145       ORAG0000  21        C02                                     0.13    143       ORAF0000  21        848                                     0.94    145       ORAE0000  27        C02                                     0.67    145       ORAG0000  21        152                                     0.94    145       ORAG0000  22        791                                     0.95    143       ORAF0000  26        890                                     0.98    141       ORAF0000  21        848                                     0.97    l55       ORAG0000  30        467                                     1.00    141       ORAG0000  21        231                                     0.95    145       ORAE0000  21        848                                     0.97    141       ORAF0000  26        CT                                      0.99    141       ORAE0000  20        354                                     0.92    155       ORAE0000  22        261                                     0.94    143       ORAF0000  28        D63                                     0.93    152       ORAF0000  29        560                                     0.90    145       ORAG0000  26        568                                     0.98    147       ORAE0000  27        C02                                     0.99    142       ORAG0000  28        090                                     0.99    314       ORAE0000  28        090                                     0.97    153       ORAG0000  29        592                                     0.92    143       ORAF0000  29        755                                     0.97    145       ORAE0000  21        848                                     0.98    143       ORAF0000  20        181                                     0.95    145       ORAG0000  28        D63                                     0.95    143       ORAG0000  20        354                                     0.98    145       ORAE0000  21        848                                     0.99    145       ORAG0000  29        D67                                     0.97    143       ORAF0000  20        595                                     0.98    143       ORAF0000  29        755                                     0.91    145       ORAE0000  28        063                                     1.00    147       ORAG0000  29        592                                     0.90    145       ORAE0000  22        215                                     0.97    145       ORAE0000  29        D67                                     0.92    143       ORAF0000  20        181                                     0.93    153       ORAF0000  22        225                                     0.88    143       ORAF0000  21        231                                     0.96    143       ORAE0000  20        595                                     0.88    157       ORAE0000  28        D65                                     0.97    143       ORAG0000  26        568                                     0.90    143       ORAG0000  26        568                                     0.98    145       ORAG0000  22        A56                                     0.95    145       ORAE0000  22        A56                                     1.00    157       ORAG0000  26        120                                     0.98    157       ORAG0000  28        C79                                     0.93    143       ORAF0000  20        354                                     0.95    143       ORAE0000  29        215                                     0.88    145       ORAE0000  26        890                                     1.00    145       ORAG0000  27        371                                     0.97    145       ORAG0000  29        D67                                     0.97    145       ORAG0000  21        426                                     0.97    145       ORAG0000  26        568                                     0.91    145       ORAE0000  20        354                                     0.96    145       ORAG0000  26        568                                     0.92    145       ORAG0000  21        261                                     0.94    145       ORAE0000  20        354                                     0.89    145       ORAG0000  22        231                                     0.97    145       ORAE0000  29        D67                                     0.97    145       ORAE0000  29        604                                     0.94    145       ORAG0000  27        B28                                     0.84    145       ORAE0000  21        426                                     1.00    145       ORAG0000  22        261                                     0.91    145       ORAG0000  27        C02                                     0.95    145       ORAE0000  20        354                                     0.96    145       ORAG0000  29        D67                                     0.93    145       ORAE0000  21        E19                                     0.98    145       ORAG0000  28        637                                     0.93    145       ORAE0000  26        CT                                      0.98    145       ORAE0000  26        568                                     0.87    145       ORAG0000  27        B28                                     0.95    145       ORAG0000  22        215                                     0.83    145       ORAG0000  27        B28                                     0.98    145       ORAG0000  29        592                                     0.67    145       ORAG0000  28        D63                                     0.82    145       ORAE0000  27        181                                     0.96    145       ORAE0000  21        848                                     0.93    145       ORAG0000  28        D63                                     0.99    145       ORAE0000  29        592                                     0.99    145       ORAG0000  29        592                                     0.96    145       ORAE0000  27        B28                                     0.97    145       ORAE0000  29        D67                                     1.00    145       ORAG0000  26        CT                                      0.78    145       ORAE0000  26        568                                     0.93    145       ORAE0000  21        426                                     0.98    145       ORAG0000  20        637                                     0.99    145       ORAG0000  20        637                                     0.94    145       ORAE0000  21        426                                     0.98    145       ORAE0000  27        595                                     0.96    145       ORAG0000  29        D67                                     0.98    145       ORAG0000  28        568                                     0.97    145       ORAE0000  21        848                                     0.98    145       ORAG0000  29        604                                     0.75    145       ORAG0000  28        D63                                     0.99    145       ORAE0000  22        261                                     0.90    145       ORAE0000  21        857                                     0.94    145       ORAE0000  27        C02                                     0.77    145       ORAE0000  22        261                                     0.94    145       ORAG0000  28        637                                     0.99    145       ORAG0000  21        E19                                     0.97    145       ORAG0000  29        604                                     0.97    145       ORAE0000  29        604                                     0.96    145       ORAE0000  28        637                                     0.95    145       ORAE0000  28        637                                     0.96    145       ORAG0000  28        637                                     0.90    145       ORAG0000  22        A56                                     0.99    145       ORAG0000  26        CT                                      6.93    145       ORAE0000  27        C02                                     0.92    145       ORAG0000  29        604                                     1.00    145       ORAG0000  28        637                                     0.89    145       ORAE0000  28        D63                                     0.97    145       ORAG0000  26        215                                     0.94    145       ORAG0000  27        C02                                     0.96    145       ORAG0000  26        568                                     0.90    145       ORAE0000  21        848                                     0.97    145       ORAG0000  22        261                                     0.97    145       ORAG0000  28        637                                     0.93    145       ORAE0000  21        857                                     0.95    145       ORAG0000  28        748                                     0.90    145       ORAG0000  27        C02                                     0.75    145       ORAG0000  27        371                                     0.97    145       ORAG0000  28        748                                     0.81    145       ORAE0000  21        857                                     1.00    145       ORAG0000  28        D63                                     1.00    145       ORAE0000  22        A56                                     0.78    145       ORAG0000  26        890                                     1.00    145       ORAG0000  21        857                                     0.68    145       ORAE0000  28        D67                                     0.82    145       ORAE0000  22        770                                     0.86    145       ORAE0000  21        848                                     0.97    145       ORAE0000  21        848                                     0.97    145       ORAE0000  20        181                                     0.91    145       ORAE0000  20        181                                     1.00    145       ORAE0000  20        181                                     1.00    145       ORAG0000  21        848                                     0.88    145       ORAE0000  26        890                                     0.94    145       ORAE0000  20        037                                     1.00    145       ORAG0000  21        E19                                     0.80    155       ORAG0000  26        890                                     0.97    155       ORAE0000  21        349                                     0.95    155       ORAG0000  20        354                                     0.97    155       ORAE0000  26        568                                     0.93    155       ORAG0000  26        568                                     0.88    155       ORAE0000  26        890                                     1.00    155       ORAG0000  29        604                                     0.90    155       ORAG0000  29        604                                     0.90    155       ORAG0000  28        755                                     0.91    155       ORAE0000  28        D05                                     0.88    155       ORAE0000  27        B28                                     0.99    155       ORAE0000  29        592                                     0.96    155       ORAE0000  21        848                                     1.00    155       ORAE0000  28        292                                     0.96    155       ORAG0000  20        431                                     0.88    155       ORAG0000  21        848                                     0.97    155       ORAE0000  28        637                                     0.92    155       ORAE0000  20        C79                                     0.97    155       ORAG0000  22        261                                     0.99    155       ORAE0000  20        C79                                     0.98    155       ORAG0000  27        595                                     0.99    155       ORAG0000  29        592                                     0.95    155       ORAG0000  22        215                                     1.00    155       ORAG0000  26        CT                                      0.90    155       ORAE0000  28        A18                                     0.98    155       ORAG0000  28        A18                                     0.99    155       ORAG0000  29        D67                                     0.95    155       ORAG0000  26        152                                     0.95    155       ORAG0000  20        C79                                     0.97    155       ORAG0000  26        568                                     0.98    155       ORAG0000  26        568                                     0.97    155       ORAG0000  28        637                                     0.89    155       ORAE0000  26        215                                     0.91    155       ORAE0000  26        D63                                     1.00    155       ORAE0000  26        890                                     0.96    155       ORAE0000  20        354                                     0.94    155       ORAE0000  28        D63                                     0.94    155       ORAG0000  22        261                                     0.95    155       ORAE0000  27        A05                                     1.00    155       ORAG0000  21        595                                     0.87    155       ORAG0000  28        292                                     0.93    155       ORAE0000  26        890                                     0.98    155       ORAG0000  28        568                                     1.00    155       ORAG0000  29        D67                                     0.99    155       ORAE0000  22        429                                     0.92    155       ORAE0000  22        429                                     0.99    153       ORAG0000  21        857                                     0,94    155       ORAG0000  20        595                                     0.97    155       ORAE0000  21        857                                     0.96    155       ORAE0000  21        857                                     1.00    l55       ORAG0000  20        181                                     1.00    155       ORAG0000  20        181                                     0.97    155       ORAE0000  21        848                                     0.72    155       ORAE0000  21        857                                     1.00    155       ORAG0000  20        181                                     0.81    155       ORAE0000  20        354                                     0.97    155       ORAG0000  22        261                                     1.00    155       ORAE0000  21        848                                     0.92    155       ORAE0000  20        181                                     1.00    155       ORAG0000  21        857                                     0.97    155       ORAE0000  21        426                                     0.61    155       ORAG0000  21        857                                     0.92    155       ORAG0000  20        595                                     1.00    155       ORAE0000  21        426                                     0.97    155       ORAE0000  20        595                                     1.00    155       ORAG0000  21        426                                     1.00    155       ORAF0000  21        426                                     0.94    155       ORAE0000  21        857                                     0.97    155       ORAE0000  20        181                                     1.00    155       ORAG0000  20        181                                     0.90    155       ORAG0000  20        595                                     0.85    155       ORAE0000  21        857                                     0.91    155       ORAG0000  21        426                                     1.00    155       ORAG0000  26        CT                                      0.94    155       ORAG0000  26        568                                     0.89    152       ORAG0000  21        848                                     1.00    152       ORAF0000  29        560                                     0.94    152       ORAE0000  22        595                                     0.96    152       ORAF0000  29        090                                     0.98    151       ORAF0000  29        604                                     1.00    151       ORAF0000  29        592                                     0.98    151       ORAF0000  26        890                                     0.98    151       ORAF0000  20        354                                     0.97    151       ORAF0000  20        354                                     0.92    151       ORAE0000  28        D67                                     0.99    151       ORAF0000  20        152                                     0.87    151       ORAG0000  20        354                                     0.98    151       ORAF0000  20        595                                     0.98    152       ORAF0000  21        857                                     0.99    152       ORAF0000  28        292                                     0.98    152       ORAE0000  26        748                                     0.99    152       ORAG0000  26        568                                     0.98    152       ORAF0000  29        592                                     0.98    157       ORAE0000  27        B28                                     1.00    157       ORAG0000  29        D67                                     1.00    157       ORAE0000  21        867                                     0.97    157       ORAF0000  28        637                                     0.99    157       ORAF0000  20        A56                                     1.00    157       ORAE0000  22        215                                     0.99    157       ORAG0000  20        354                                     1.00    157       ORAE0000  26        568                                     0.99    157       ORAF0000  21        848                                     1.00    151       ORAF0000  21        467                                     1.00    151       ORAG0000  20        354                                     0.97    151       ORAG0000  28        D67                                     0.96    153       ORAF0000  20        181                                     0.87    153       ORAF0000  22        C02                                     0.91    143       ORAF0000  28        637                                     0.86    143       ORAG0000  27        D67                                     0.70    143       ORAF0000  21        D67                                     0.93    143       ORAE0000  22        C02                                     0.93    143       ORAF0000  26        CT                                      0.90    143       ORAG0000  26        CT                                      0.92    143       ORAE0000  29        755                                     0.89    143       ORAF0000  28        637                                     0.75    143       ORAE0000  21        848                                     0.94    143       ORAF0000  22        215                                     0.92    143       ORAF0000  20        E19                                     0.96    143       ORAF0000  27        B28                                     0.94    143       ORAF0000  29        604                                     0.83    143       ORAF0000  290       604                                     0.92    142       ORAE0000  22        261                                     0.94    141       ORAE0000  20        595                                     0.99    142       ORAF0000  29        604                                     0.91    142       ORAF0000  27        748                                     0.94    141       ORAG0000  20        595                                     0.99    141       ORAF0000  22        090                                     0.98    141       ORAG0000  22        090                                     0.97    141       ORAF0000  21        800                                     0.92    141       ORAE0000  20        354                                     0.96    141       ORAF0000  27        C02                                     0.96    141       ORAE0000  27        B28                                     0.97    141       ORAE0000  29        D67                                     0.97    141       ORAF0000  27        890                                     0.99    141       ORAF0000  22        261                                     0.98    141       ORAF0000  21        857                                     0.91    141       ORAE0000  20        354                                     0.94    141       ORAE0000  20        354                                     0.98    141       ORAF0000  27        595                                     0.93    141       ORAE0000  26        890                                     0.97    141       ORAE0000  21        857                                     1.00    141       ORAF0000  27        C02                                     0.98    141       ORAG0000  26        890                                     0.98    141       ORAF0000  20        354                                     0.98    142       ORAG0000  21        426                                     0.99    142       ORAG0000  27        C02                                     0.95    142       ORAF0000  21        231                                     1.00    142       ORAF0000  20        431                                     0.99    142       ORAF0000  20        354                                     0.93    142       ORAF0000  29        604                                     0.95    142       ORAF0000  20        181                                     0.84    142       ORAF0000  22        261                                     0.96    142       OPAF0000  21        595                                     0.94    142       ORAG0000  20        215                                     0.99    142       ORAG0000  26        890                                     0.92    142       ORAF0000  29        592                                     0.83    142       ORAF0000  21        426                                     0.97    142       ORAG0000  27        B28                                     0.86    142       ORAF0000  21        426                                     0.96    142       ORAF0000  27        B28                                     0.93    142       ORAG0000  20        090                                     0.99    142       ORAE0000  28        D63                                     0.90    142       ORAF0000  26        890                                     0.99    142       ORAF0000  29        592                                     0.99    142       ORAG0000  29        592                                     0.99    142       ORAE0000  22        261                                     1.00    142       ORAE0000  27        A05                                     0.95    142       ORAF0000  20        215                                     1.00    142       ORAF0000  26        CT                                      0.93    142       ORAE0000  28        292                                     0.97    142       ORAE0000  21        426                                     0.99    147       ORAG0000  21        426                                     0.99    147       ORAG0000  29        592                                     0.99    147       ORAE0000  21        426                                     0.94    147       ORAG0000  28        292                                     0.99    147       ORAF0000  29        D67                                     0.98    147       ORAE0000  20        354                                     0.96    148       ORAG0000  20        604                                     0.96    148       ORAG0000  20        354                                     0.93    142       ORAE0000  27        181                                     0.91    142       ORAF0000  29        604                                     0.97    142       ORAG0000  29        604                                     0.96    142       ORAE0000  22        231                                     1.00    142       ORAG0000  26        890                                     0.99    142       ORAE0000  20        354                                     0.89    143       ORAF0000  21        231                                     0.87    143       ORAF0000  22        215                                     0.96    143       ORAE0000  28        292                                     0.97    143       ORAG0000  20        848                                     0.92    143       ORAG0000  20        595                                     0.96    143       ORAF0000  21        848                                     0.89    143       ORAF0000  26        568                                     0.91    143       ORAF0000  22        215                                     0.88    143       ORAG0000  28        292                                     0.94    143       ORAG0000  29        755                                     0.94    143       ORAF0000  22        215                                     0.97    143       ORAG0000  20        354                                     0.93    143       ORAF0000  22        215                                     0.92    143       ORAF0000  22        231                                     0.94    143       OPAE0000  22        231                                     0.89    215       ORAF0000  30        467                                     0.47    155       ORAE0000  30        467                                     1.00    157       ORAG0000  29        D67                                     0.96    145       ORAE0000  28        568                                     0.88    155       ORAE0000  20        354                                     0.91    143       ORAE0000  26        261                                     0.95    155       ORAE0000  28        292                                     0.89    145       ORAG0000  27        595                                     0.99    155       ORAG0000  29        592                                     0.98    155       ORAE0000  20        595                                     0.99    155       ORAG0000  28        568                                     0.96    145       ORAE0000  27        E86                                     0.95    155       ORAE0000  26        261                                     0.92    155       ORAE0000  26        D65                                     0.93    143       ORAE0000  27        604                                     0.99    155       ORAE0000  28        568                                     1.00    155       ORAG0000  28        292                                     1.00    145       ORAG0000  26        CT                                      0.98    148       ORAG6000  26        CT                                      0.99    145       ORAE0000  27        604                                     0.99    142       ORAE0000  27        181                                     0.91    142       ORAG0000  21        857                                     0.94    155       ORAE0000  29        890                                     1.00    145       ORAE0000  27        604                                     0.97    145       ORAG0000  29        C02                                     0.97    145       ORAG0000  28        E19                                     0.87    155       ORAE0000  27        595                                     0.95    145       ORAG0000  27        748                                     0.95    145       ORAE0000  28        E19                                     0.95    152       ORAE0000  20        231                                     0.99    142       ORAG0000  29        592                                     0.98    145       ORAG0000  20        637                                     0.95    155       ORAG0000  20        354                                     0.99    142       ORAE0000  27        604                                     0.99    145       ORAE0000  27        604                                     0.99    145       ORAE0000  28        D63                                     0.99    142       ORAE0000  21        426                                     1.00    145       ORAE0000  29        D67                                     1.00    141       ORAG0000  878       292                                     1.00    152       ORAG0000  26        CT                                      0.98    145       ORAE0000  29        568                                     0.97    145       ORAG0000  27        152                                     0.90    142       ORAG0000  26        890                                     0.95    145       ORAE0000  28        D63                                     1.00    145       ORAG0000  29        568                                     0.97    145       ORAE0000  27        152                                     0.95    143       ORAE0000  29        C02                                     0.96    145       ORAE0000  27        637                                     0.95    145       ORAE0000  27        152                                     1.00    145       ORAG0000  29        D67                                     0.98    145       ORAG0000  28        568                                     1.00    143       ORAG0000  29        568                                     0.94    143       ORAG0000  28        292                                     0.96    142       ORAG0000  29        604                                     0.89    145       ORAE0000  27        637                                     0.98    145       ORAG0000  29        592                                     0.83    145       ORAG0000  27        637                                     0.78    143       ORAE0000  27        748                                     0.97    145       ORAG0000  28        D63                                     0.87    145       ORAG0000  27        152                                     0.95    145       ORAG0000  27        604                                     1.00    145       ORAG0000  28        568                                     1.00    145       ORAE0000  28        568                                     O.97    145       ORAE0000  28        568                                     0.96    142       ORAE0000  29        568                                     0.94    145       ORAG0000  28        568                                     0.98    145       ORAG0000  26        261                                     1.00    145       ORAG0000  27        604                                     1.00    145       ORAG0000  27        512                                     0.95    145       ORAG0000  27        604                                     0.95    145       ORAG0000  27        604                                     1.00    145       ORAG0000  28        568                                     1.00    145       ORAG0000  28        568                                     0.99    148       ORAG0000  26        CT                                      0.94    145       ORAE0000  27        604                                     0.92    145       ORAG0000  26        CT                                      1.00    145       ORAG0000  26        261                                     1.00    145       ORAE0000  28        592                                     0.97    145       ORAE0000  28        592                                     0.94    145       ORAG0000  28        D63                                     0.97    145       ORAG0000  26        890                                     0.97    145       ORAE0000  28        568                                     0.96    145       ORAG0000  26        261                                     0.91    145       ORAE0000  27        B28                                     0.91    145       ORAG0000  27        B28                                     1.00    145       ORAE0000  27        B28                                     1.00    145       ORAE0000  27        371                                     1.00    145       ORAG0000  28        568                                     1.00    145       ORAG0000  28        592                                     0.92    145       ORAG0000  29        592                                     0.94    145       ORAG0000  29        592                                     0.98    142       ORAG0000  29        D67                                     0.91    145       ORAE0000  29        D67                                     1.00    145       ORAG0000  26        890                                     0.94    145       ORAG0000  28        D63                                     0.94    145       ORAE0000  28        D63                                     0.94    145       ORAG0000  27        604                                     0.97    148       ORAG0000  26        261                                     1.00    145       ORAG0000  26        890                                     0.93    143       ORAE0000  27        604                                     0.97    145       ORAG0000  29        592                                     1.00    145       ORAE0000  28        568                                     0.95    145       ORAE0000  27        604                                     0.99    148       ORAE0000  29        592                                     0.93    143       ORAE0000  28        D63                                     1.00    145       ORAG0000  26        890                                     0.92    145       ORAG0000  29        C02                                     0.74    145       ORAE0000  29        C02                                     1.00    145       ORAE0000  26        890                                     1.00    145       ORAG0000  29        D67                                     1.00    145       ORAE0000  29        D67                                     0.97    145       ORAG0000  27        371                                     0.65    145       ORAG0000  28        D63                                     0.99    145       ORAG0000  29        C02                                     1.00    145       ORAG0000  28        568                                     1.00    155       ORAE0000  28        292                                     0.81    145       ORAG0000  28        568                                     0.97    143       ORAE0000  28        292                                     0.98    145       ORAG0000  28        568                                     0.99    148       ORAG0000  28        D63                                     0.98    142       ORAE0000  29        152                                     0.96    145       ORAE0000  29        592                                     0.94    145       ORAG0000  29        C02                                     0.96    145       ORAG0000  28        568                                     1.00    145       ORAE0000  29        C02                                     0.98    145       ORAG0000  28        292                                     0.87    145       ORAE0000  27        604                                     0.81    145       ORAE0000  27        371                                     0.93    145       ORAE0000  27        B28                                     0.99    145       ORAE0000  26        CT                                      0.95    145       ORAE0000  26        CT                                      0.99    145       ORAE0000  26        CT                                      0.86    145       ORAG0000  27        371                                     0.97    145       ORAE0000  29        592                                     0.92    145       ORAG0000  28        D63                                     0.97    143       ORAG0000  26        890                                     0.99    145       ORAG0000  26        261                                     0.95    145       ORAE0000  28        568                                     0.98    145       ORAG0000  29        D67                                     1.00    145       ORAE0000  29        D67                                     0.97    143       ORAG0000  28        D67                                     0.94    143       ORAG0000  26        OCT                                     0.97    147       ORAG0000  27        B28                                     0.99    145       ORAE0000  28        D63                                     0.99    145       ORAG0000  28        568                                     0.98    145       ORAE0000  26        CT                                      0.95    145       ORAG0000  26        261                                     0,96    145       ORAE0000  27        604                                     1.00    155       ORAG0000  26        890                                     0.99    155       ORAE0000  29        568                                     0.97    155       ORAG0000  29        D67                                     0.99    155       ORAE0000  28        568                                     0.98    148       ORAE0000  28        292                                     0.98    143       ORAE0000  28        D63                                     0.97    143       ORAE0000  27        604                                     0.99    155       ORAG0000  28        292                                     0.96    143       ORAE0000  29        592                                     1.00    152       ORAG0000  28        D63                                     1.00    155       ORAE0000  26        849                                     0.97    145       ORAE0000  26        890                                     1.00    145       ORAG0000  26        CT                                      0.94    145       ORAE0000  28        568                                     0.84    145       ORAG0000  28        568                                     0.96    142       ORAG0000  29        592                                     0.99    145       ORAE0000  29        592                                     0.95    142       ORAE0000  26        CT                                      0.89    142       ORAE0000  27        B28                                     0.97    142       ORAE0000  28        D63                                     0.81    142       ORAE0000  26        890                                     0.94    148       ORAE0000  878       292                                     0.95    143       ORAG0000  29        592                                     0.93    155       ORAG0000  28        568                                     0.98    142       ORAG0000  29        C02                                     0.99    142       ORAE0000  29        090                                     0.94    155       ORAG0000  29        592                                     0.94    143       ORAG0000  28        D63                                     0.99    145       ORAG0000  29        C02                                     0.99    155       ORAG0000  28        568                                     0.92    153       ORAE0000  27        604                                     0.98    153       ORAG0000  28        568                                     1.00    155       ORAG0000  876       E19                                     0.94    143       ORAE0000  27        B28                                     0.94    145       ORAE0000  29        090                                     0.94    143       ORAE0000  20        637                                     0.95    145       ORAG0000  27        604                                     0.99    142       ORAG0000  29        592                                     0.92    143       ORAE0000  27        B28                                     0.96    145       ORAG0000  876       CT                                      0.99    155       ORAE0000  29        592                                     0.94    147       ORAG0000  28        292                                     0.91    153       ORAE0000  877       B28                                     0.97    152       ORAG0000  28        637                                     1.00    155       ORAG0000  26        261                                     0.99    142       ORAG0000  879       890                                     0.94    145       ORAG0000  879       090                                     0.95    143       ORAE0000  29        592                                     0.94    145       ORAG0000  876       261                                     0.96    153       ORAG0000  877       748                                     0.96    145       ORAE0000  876       CT                                      0.98    155       ORAE0000  879       C02                                     0.95    155       ORAG0000  26        261                                     0.92    155       ORAE0000  876       CT                                      0.92    155       ORAG0000  28        770                                     1.00    142       ORAG0000  879       890                                     0.93    155       ORAG0000  877       604                                     0.98    145       ORAG0000  879       592                                     0.94    143       ORAE0000  677       748                                     0.97    145       ORAE0000  878       770                                     0.88    147       ORAE0000  876       CT                                      0.97    155       ORAE0000  879       770                                     0.95    145       ORAE0000  890       890                                     0.97    143       ORAE0000  26        890                                     0.96    145       ORAE0000  877       604                                     0.97    145       ORAE0000  28        568                                     0.59    153       ORAE0000  876       152                                     0.96    142       ORAE0000  28        292                                     0.94    153       ORAE0000  877       748                                     0.97    153       ORAE0000  879       C02                                     0.98    145       ORAG0000  878       D63                                     0.99    152       ORAG0000  878       D63                                     1.00    145       ORAG0000  28        568                                     0.98    152       ORAG0000  21        C02                                     0.83    145       ORAE0000  877       770                                     0.97    145       ORAE0000  879       592                                     0.84    145       ORAE0000  877       770                                     0.96    141       ORAG0000  876       152                                     0.99    141       ORAG0000  876       261                                     0.96    145       ORAE0000  879       890                                     0.99    145       ORAE0000  879       C02                                     0.99    142       ORAE0000  20        354                                     0.94    155       ORAG0000  878       D63                                     0.80    145       ORAE0000  876       CT                                      0.88    145       ORAE0000  877       A18                                     0.88    143       ORAG0000  29        592                                     0.91    143       ORAG0000  877       B28                                     0.85    143       ORAE0000  877       A18                                     0.91    152       ORAE0000  877       604                                     0.99    145       ORAE0000  878       568                                     0.80    145       ORAG0000  877       604                                     0.93    145       ORAE0000  879       890                                     0.94    145       ORAE0000  878       748                                     0.98    147       ORAE0000  876       D77                                     0.90    143       ORAE0000  878       568                                     1.00    145       ORAE0000  878       568                                     0.92    141       ORAE0000  26        261                                     0.97    145       ORAE0000  29        SM                                      0.94    145       ORAG0000  87        604                                     0.95    143       ORAE0000  878       568                                     0.99    145       ORAE0000  878       568                                     0.92    145       ORAG0000  879       879                                     0.98    145       ORAE0000  878       568                                     1.00    155       ORAG0000  879       SM                                      0.95    145       ORAG0000  879       C02                                     0.97    155       ORAE0000  879       SM                                      0.96    155       ORAG0000  879       770                                     0.97    155       ORAG0000  878       568                                     0.96    145       ORAE0000  879       890                                     0098    142       ORAG0000  29        748                                     0.99    147       ORAE0000  876       D67                                     1.00    145       ORAE0000  879       890                                     0.99    145       ORAG0000  877       604                                     0.96    155       ORAE0000  878       D63                                     0.96    143       ORAE0000  29        SM                                      0.76    145       ORAE0000  877       152                                     0.97    145       ORAG0000  877       604                                     1.00    155       ORAG0000  879       371                                     1.00    155       ORAG0000  878       349                                     1.00    155       ORAE0000  879       SM                                      0.97    155       ORAG0000  877       C09                                     0.85    155       ORAE0000  877       090                                     0.99    145       ORAG0000  879       SM                                      0.86    145       ORAG0000  876       D77                                     0.96    145       ORAE0000  29        C02                                     0.99    145       ORAG0000  878       568                                     0.99    145       ORAE0000  878       568                                     0.98    145       ORAE0000  877       604                                     0.94    145       ORAE0000  877       D67                                     0.97    145       ORAE0000  876       261                                     0.95    145       ORAE0000  878       D63                                     0.99    145       ORAE0000  878       568                                     1.00    145       ORAG0000  29        C02                                     1.00    145       ORAE0000  876       D77                                     1.00    145       ORAG0000  879       SM                                      0.97    155       ORAE0000  876       CT                                      0.91    155       ORAE0000  879       SM                                      0.63    155       ORAG0000  876       D77                                     0.97    155       ORAE0000  876       261                                     1.00    155       ORAE0000  878       568                                     1.00    155       ORAE0000  876       CT                                      0.94    155       ORAE0000  877       D67                                     0.97    155       ORAG0000  878       568                                     0.96    155       ORAG0000  877       B28                                     1.00    155       ORAG0000  878       568                                     0.93    155       ORAE0000  877       D67                                     1.00    155       ORAG0000  878       292                                     0.95    155       ORAG0000  876       261                                     0.95    155       ORAG0000  879       SM                                      1.00    155       ORAG0000  879       879                                     0.96    143       ORAE0000  877       B28                                     0.93    145       ORAG0000  877       C09                                     0.99    145       ORAG0000  878       568                                     0.95    143       ORAE0000  877       604                                     0.99    152       ORAE0000  878       C02                                     0.97    145       ORAE0000  877       604                                     0.97    155       ORAG0000  877       604                                     0.96    145       ORAG0000  879       261                                     1.00    145       ORAE0000  876       152                                     0.96    143       ORAE0000  877       604                                     0.97    145       ORAG0000  879       SM                                      0.98    155       ORAE0000  876       261                                     0.94    145       ORAE0000  879       SM                                      0.94    145       ORAE0000  878       292                                     0.83    145       ORAG0000  877       604                                     0.92    145       ORAE0000  877       604                                     0.98    145       ORAE0000  879       604                                     0.97    145       ORAG0000  878       349                                     0.93    153       ORAG0000  877       604                                     1.00    145       ORAE0000  879       SM                                      0.97    145       ORAE0000  878       568                                     0.93    145       ORAE0000  876       D63                                     0.76    145       ORAG0000  877       748                                     0.97    142       ORAE0000  29        592                                     0.91    143       ORAG0000  878       C02                                     0.96    143       ORAE0000  878       568                                     1.00    145       ORAE0000  878       568                                     0.98    155       ORAE0000  876       CT                                      0.98    155       ORAG0000  876       261                                     0.89    143       ORAE0000  877       B28                                     0.86    145       ORAE0000  877       152                                     0.41    143       ORAG0000  877       152                                     0.96    145       ORAE0000  877       748                                     ______________________________________                                    

The first column in Table 1 contains yield information in terms ofmeasure signals 212 that have been normalized by processor 214. Columns2-5 contain an alpha-numeric representation of ID signals 208 for theindividual attributes 1 through K. Each row of Table 1 represents aparticular manufacturing run; therefore, the normalized value of themeasure signal 212 appearing in a row is a yield for that particularmanufacturing run and the alphanumeric representations of the ID signals208 indicate the individual attributes that contributed to thatparticular manufacturing run.

The convergent weighted averages for each of the attributes aredescribed in Table 2.

                                      TABLE 2                                     __________________________________________________________________________    Attributes analysis of YIELD vs MEFAM, TOOL, SCREENER and OPID.               Cleaned up SAMPLE DATA data file contained 836 records.                       Yields quoted are self-consistent, weighted avarages.                         MEFAM                                                                              YIELD                                                                              TOOL  YIELD                                                                              SCREENER                                                                             YIELD                                                                              OPID                                                                              YIELD                                    __________________________________________________________________________    314  0.99 ORAG0000                                                                            0.947                                                                              878    0.972                                                                              120 1                                        157  0.983                                                                              ORAE0000                                                                            0.94 879    0.971                                                                              867 1                                        147  0.975                                                                              ORAF0000                                                                            0.933                                                                              29     0.963                                                                              512 1                                        148  0.973           28     0.951                                                                              849 1                                        151  0.973           890    0.95 431 0.98                                     152  0.967           26     0.948                                                                              680 0.98                                     141  0.958           20     0.945                                                                              004 0.98                                     142  0.953           87     0.94 349 0.975                                    155  0.947           27     0.939                                                                              592 0.972                                    145  0.942           23     0.935                                                                              SM  0.972                                    143  0.913           22     0.931                                                                              D6J 0.97                                     153  0.899           876    0.926                                                                              800 0.97                                     215  0.89            21     0.918                                                                              560 0.968                                                         877    0.899                                                                              E19 0.966                                                         290    0.83 A05 0.965                                                         30     0.779                                                                              568 0.964                                                                     A56 0.964                                                                     CT  0.962                                                                     E86 0.96                                                                      879 0.96                                                                      292 0.96                                                                      429 0.955                                                                     637 0.955                                                                     C02 0.954                                                                     D67 0.951                                                                     261 0.949                                                                     595 0.948                                                                     090 0.947                                                                     C09 0.947                                                                     181 0.945                                                                     604 0.944                                                                     426 0.942                                                                     791 0.94                                                                      037 0.94                                                                      OCT 0.94                                                                      354 0.939                                                                     215 0.937                                                                     225 0.936                                                                     231 0.936                                                                     CO2 0.935                                                                     890 0.935                                                                     371 0.933                                                                     D63 0.933                                                                     755 0.932                                                                     B28 0.932                                                                     669 0.93                                                                      C79 0.928                                                                     097 0.925                                                                     748 0.926                                                                     A18 0.918                                                                     555 0.915                                                                     569 0.912                                                                     063 0.91                                                                      D05 0.91                                                                      857 0.908                                                                     233 0.907                                                                     D65 0.9                                                                       770 0.899                                                                     848 0.892                                                                     152 0.872                                                                     391 0.87                                                                      D77 0.867                                                                     432 0.86                                                                      255 0.86                                                                      467 0.826                                    __________________________________________________________________________

For each individual ID signal 208 a self-consistent weighted average hasbeen calculated and is displayed. The report in Table 2 is sorted on IDsignal 208 and arranged in order of "best" individual attribute to"worst" individual attribute. For example, screener ID 878 has aself-consistent weighted average of 0.972 while screener 30 has aself-consistent weighted average of 0.779. This tells the reader of thisreport that, independent of the variations in and contributions of theother attributes in the manufacturing run, screener 878 is a higherquality screener than screener 30.

FIG. 3 is an example of a graphical representation of theself-consistent weighted averages of each of the individual attributes.This provides an at-a-glance review of the quality of each attributewith respect to each of the other attributes in its class.

The preferred embodiment can also be described with the aid ofmathematical expressions. Each measure Y_(i) is the product of theweights of all k of its attributes. This can be shown by the expression:##EQU1## where W_(ij)(i) is the weight of the j_(i) th member of the ithattribute and Y_(i) is normalized between epsilon and 1 where epsilon isa small positive number (say 10⁻¹³). The use of an epsilon obviates theneed to treat 0 weights.

The iteration equation for computing the attributes' weights is ##EQU2##where Y_(mj)(i) is the mth measure associated with W_(ij)(i) andW'_(n),ij(i) is the nth attribute weight associated with Y_(mj)(i) withthe exception of W_(ij)(i) itself.

The preferred embodiment describes a correlation given by the ratio ofcurrent attribute weight to the product of all other weights associatedwith a given normalized measure. The new attribute weight is theweighted average of all associated performance measures.

An alternative embodiment can be described for computing correlationsgiven by the ratio of current attribute weight to the sum of all otherweights associated with the given normalized measure. The new attributeweight is the weighted sum of all associated performance measures.

The alternative embodiment also can be described with the aid ofmathematical expressions. In the alternative embodiment, each measureY_(i) is the sum of the weight of all k of its attributes. This can beshown by the expression: ##EQU3## where W_(ij)(i) is the weight of the_(j)(i) the member of the ith attribute.

The iteration equation for the alternative method is ##EQU4##

Typically, the first embodiment converges faster. The selection of thecorrelation in the weighted sum guarantees fast convergence due to itshighly non-linear structure which highlights the relative contributionsin the selected ratio. Oscillations, that occur in other iterativetechniques, do not occur here due to the smoothing effects of theweighted sum.

An embodiment of the present invention is illustrated by the source codelisting which appears in the Appendix.

3. Conclusion

While various embodiments of the present invention have been describedabove, it should be understood that they have been presented by way ofexample only, and not limitation. Thus, the spirit and scope of thepresent invention should not be limited by any of the above-describedexemplary embodiments, but should be defined only in accordance with thefollowing claims and their equivalents.

Having thus described our invention, what we claim as new and desire tosecure by Letters Patent is:
 1. A program storage device readable by amachine, tangibly embodying a program of instructions executable by themachine to perform method steps for quantitatively ranking theperformance of a plurality of non-numeric attributes of a manufacturingline, said method steps comprising:(a) recording in a computer processordatabase a measure signal for each of a plurality of manufacturing runson the manufacturing line, wherein each said measure signal comprises adata signal representing an output measure of the manufacturing line,and said data signals are obtained by operating the manufacturing lineto produce a product; (b) recording in said computer processor databaseattribute ID signals associated with each of said plurality ofmanufacturing runs, wherein each of said attribute ID signals identifiesa non-numeric attribute that contributed to the manufacturing run; (c)normalizing said recorded measure signal for each of said plurality ofmanufacturing runs; and (d) using a convergent iterative process todeconvolute said normalized recorded measure signals and said attributeID signals to generate a quantitative numeric measure signal for thenon-numeric attributes.
 2. The program storage device of claim 1,wherein said method step (d) further comprises:(i) selectingmanufacturing runs to which a selected one of the plurality ofnon-numeric attributes contribute; (ii) ascribing said recorded measuresignal for each selected run to said selected non-numeric attribute;(iii) determining a weighted average of all said ascribed measuresignals to generate said quantitative numeric measure signal for saidselected non-numeric attribute; and (iv) repeating said steps (i)-(iii)for each of the plurality of non-numeric attributes to generate saidquantitative numeric measure signal for each of the plurality ofnon-numeric attributes.
 3. The program storage device of claim 2,further comprising the method steps of:(v) computing a refinedquantitative numeric measure signal for a selected one of the pluralityof non-numeric attributes as a ratio between said normalized measuresignal and a product of said quantitative numeric measure signals of allother non-numeric attributes; and (vi) repeating said steps (i)-(iii)for each of the plurality of non-numeric attributes to generate saidquantitative numeric measure signal for each of the plurality ofnon-numeric attributes.
 4. The program storage device of claim 3,further comprising the method step of:(vii) repeating said steps(v)-(vi) until said refined quantitative numeric measure signalsconverge.
 5. The program storage device of claim 1, wherein said methodstep (c) comprises subtracting a value ε from said recorded measuresignal and dividing this result by a maximum said recorded measuresignal to normalize said recorded measure signal of each manufacturingrun to value between ε and one.
 6. The program storage device of claim1, further comprising the method step of:(e) editing manufacturing runscontaining anomalous data points.
 7. The program storage device of claim1, further comprising the method step of:(e) adjusting the non-numericattributes based on said quantitative numeric measure signal to optimizethe manufacturing line, wherein production of said product is optimizedas a result of said adjusting.
 8. A computer program product forquantitatively determining the performance of a plurality of non-numericattributes of a manufacturing line, said computer program productcomprising:a computer usable medium having computer readable programcode means embodied in said medium for quantitatively determining theperformance of a plurality of non-numeric attributes of a manufacturingline, said computer readable program code means having: computerreadable program code means for causing a computer to record in acomputer processor database a measure signal for each of a plurality ofmanufacturing runs on the manufacturing line, wherein each said measuresignal comprises a data signal representing an output measure of themanufacturing line and said data signals are obtained by operating themanufacturing line to produce a product; computer readable program codemeans for causing said computer to record attribute ID signalsassociated with each of said plurality of manufacturing runs, whereineach of said attribute ID signals identifies a non-numeric attributethat contributed to the manufacturing run; computer readable programcode means for causing said computer to normalize said recorded measuresignal for each of said plurality of manufacturing runs; and computerreadable program code means for causing said computer to deconvolutesaid normalized recorded measure signals and attribute ID signals usinga convergent iterative process to generate a quantitative numericmeasure signal for the non-numeric attributes.
 9. The computer programproduct of claim 8, wherein said computer readable program code meansfor causing said computer to deconvolute comprises:computer readableprogram code means for causing said computer to select manufacturingruns to which a selected one of the plurality of non-numeric attributescontributed; computer readable program code means for causing saidcomputer to ascribe said recorded measure signal for each selected runto said selected non-numeric attribute; and computer readable programcode means for causing said computer to determine a weighted average ofall said ascribed measure signals to generate said quantitative numericmeasure signal for said selected non-numeric attribute.
 10. The computerprogram product of claim 9, wherein said computer readable program codemeans for causing said computer to deconvolute further comprises meansfor computing a refined quantitative numeric measure signal for aselected one of the plurality of non-numeric attributes as a ratiobetween said normalized measure signal and a product of saidquantitative numeric measure signals of all other non-numericattributes.
 11. The computer program product of claim 8, wherein saidcomputer readable program code means for causing said computer tonormalize comprises computer readable program code means for causingsaid computer to edit manufacturing runs containing anomalous datapoints.